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ABSTRACT 



Quest is a school -improvement program based on principles of 
inquiry, collaboration, and action research. Quest supports and investigates 
ongoing school -improvement efforts through rallies, summer symposia, a 
Scholars program, visits to participating schools, communication via listserv 
and mailings, and the creation of a Quest network of schools. This report 
discusses an exploratory, multi-method study of the factors a sample of Quest 
participants believed had been important to their schools' initial and 
sustained involvement with the project. Six focus -group interviews were 
conducted with 41 Quest network members, and 26 telephone interviews and 58 
surveys concerning factors affecting engagement were completed. To compare 
schools' levels of involvement in the network, ratings were assigned each 
school, indicating -an assessment of the extent of its engagement. Elementary 
and high school data were also compared. In terms of qualitative results, 
administrative support- -both logistical and moral- -was cited most often as 
the factor most important to initial and sustained engagement with Quest. 

Also important were: the reliability of funding to support participation; the 
relevance of Quest to schools' current concerns, goals, and beliefs; and the 
schoolwide consensus for involvement in the network. Elementary schools rated 
more highly than high schools in the importance of time to do improvement 
work at their schools, the amount of time available for professional 
development, and the role of vision for learning as a factor in their 
schools' initial involvement in Quest. Six items are appended, including the 
Focus Group Protocol, the Quest Engagement Survey, and the Completed 
Evaluation Standards Checklist. (DFR) 
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EXECUTIVE SUMMARY 



After a year and a half of nurturing a continuous school improvement network, staff of the 
Quest project at AEL became interested in what factors impact engagement with school reform. 
Based upon principles of inquiry, collaboration, and action research, Quest proposes to support and 
investigate ongoing school improvement efforts through twice-yearly conferences (which staff 
renamed rallies), summer symposia, a Scholars program, visits to participating schools, 
communication via listserv and mailings, and the creation of a Quest network of schools (see 
Appendix A). As the project evolved between 1997 and 1999 (Howley-Rowe, 1998a-g; 1999a-c), 
Quest staff observed that some school teams attended project events consistently, contributed 
frequently to network activities, and remained in regular contact. Other schools continued their 
involvement, but with less intensity, while a third group of schools chose not to maintain their 
participation with the network. Quest staff also recalled the difficulty with which they had recruited 
high schools and hypothesized that there were differences between school levels in their ability to 
engage with school reform. 

This report discusses an exploratory, multimethod study conducted by Quest staff of the 
factors a sample of Quest participants believed had been important to their schools’ initial and 
sustained involvement with the project. Six focus group interviews were conducted with 41 Quest 
network members; telephone interviews were conducted with 26 less engaged participants; and 58 
surveys concerning factors affecting engagement were completed and returned to Quest staff. To 
compare differences in ratings of factors by participants hailing from various schools with disparate 
levels of involvement in the network, ratings were assigned each school indicating an assessment 
of the extent of its engagement. In addition, elementary and high school data were compared. 

In terms of qualitative results, administrative support, both logistical and moral, was cited 
most often as the factor most important to initial and sustained engagement with Quest. Also 
important were the availability of funding to support participation, relevance of Quest to schools’ 
current concerns, goals, and beliefs, and schoolwide consensus for involvement in the network. 

Tests of statistical significance were conducted, although their assumptions were violated in 
this study, in order to provide exploratory insights. T-tests indicated that elementary schools, at 
statistically significant levels, rated more highly than high schools the importance of the amount of 
time needed to do improvement work at their schools, the amount of time available for professional 
development, and the vision for learning in their schools to their schools’ initial involvement in 
Quest. In terms of sustained engagement, at statistically significant levels elementary schools 
continued to rate more highly the amount of time needed to do improvement work and available for 
professional development. The importance of the ways in which Quest supports schools 
improvement, such as via rallies and co-ventures, to continued involvement was rated statistically 
significantly higher by elementary schools, as well. 



Analyses of variance (ANOVA) located 12 statistically significant differences in ratings 
between the most engaged, moderately, and least engaged schools. For instance, highly involved 
schools found personal beliefs about education and change, the ways Quest supports school 
improvement, and Quest processes statistically significantly more important to their schools’ initial 
involvement than did the least engaged schools. The scheduling of Quest events, relevance of Quest 
to the schools, and the match between school and Quest network goals, among other factors, were 
rated more important to sustained involvement by highly involved schools than by both moderately 
and least engaged schools. 

Concluding remarks suggest that there are several important differences in the significance 
of various factors to elementary and high schools, and to the most, moderately, and least involved 
Quest schools. Recommendations include courting administrative support for school reform 
initiatives in various ways, ensuring that Quest remain relevant to current school concerns and 
undertakings, and offering support to those schools confronting logistical obstacles to network 
participation. In addition, it is recommended that further and more rigorous study of engagement 
be conducted. 



INTRODUCTION 



School improvement is increasingly viewed as an ongoing and comprehensive process. 
Recent legislation has encouraged the adoption of such a view, with the 1998 appropriation of $150 
million by Congress to states for allocation to schools undertaking research-based schoolwide reform 
programs through the Comprehensive School Reform Demonstration Program (CSRD). Earlier, in 
1994, Congress altered regulations to allow schools receiving Title I funds, with free and reduced 
lunch 50% and above, to use such funds for whole school improvement (American Institutes for 
Research, 1999). 

The reform models mentioned in the legislation instituting CSRD encompass a variety of 
approaches to reform, from skill-based, to comprehensive, to processual. In addition, the models 
vary in their degree of prescriptiveness. All claim to be based upon research and to have evidence 
of some positive impact. Yet investigations of and prototypes for school improvement extend far 
beyond the models forwarded in CSRD legislation: Contemporary literature on school improvement 
has roots in the school effectiveness literature of the 1970s and early 80s (Levine & Lezotte, 1995). 

Much current research suggests that the interplay between school cultural and structural 
conditions significantly affects how change at a particular school will be greeted (e.g., Newmann & 
Wehlage, 1996). They contend that if cultural characteristics, such as commitment to high 
expectations, support for inquiry, and caring relationships, intersect with structural factors, such as 
time for staff development and freedom from excessive organizational constraints, school reform 
will proceed more smoothly. Along with these intersections, school leadership must be an integral 
part of improvement efforts (van der Bogert, 1998), and collaboration among the many stakeholders 
in school communities must be pursued (Sarason & Lorentz, 1998). Fullan and Miles (1994) 
additionally suggest that those involved in improvement must recognize that it is a journey, one 
filled with ambiguity, uncertainty, and risk, rather than a scripted, easily implemented recipe. 

While an abundance of education research has focused on what practices and conditions 
contribute to continuous school improvement (e.g., American Institutes for Research, 1999; Barth, 
1990; Fullan, 1991; Kruse, Louis, & Bryk, 1994; Newmann, 1996; Sergiovanni, 1994), relatively 
less attention has been given to the reasons schools are willing or able to maintain their engagement 
with a particular strategy or program over time. For example, Slavin, Dolan, and Madden (1994, 
p. 30) argue that, "To survive the inevitable changes of superintendents, principals, teachers, and 
district policies, school staffs need to feel that there is a valued and important group beyond the 
confines of their district that cares and supports what they are doing." Yet the very participation of 
a school in such a group beyond their district may depend substantially on the moral and financial 
support of those within the district. 

Staff of the Quest project at AEL were especially interested in this issue after a year and a 
half of nurturing a continuous school improvement network. Based upon principles of inquiry, 
collaboration, and action research, Quest proposes to support and investigate ongoing school 
improvement efforts through twice-yearly conferences (which staff renamed rallies), summer 
symposia, a Scholars program, visits to participating schools, communication via listserv and 



mailings, and the creation of a Quest network of schools (see Appendix A). As the project evolved 
between 1997 and 1999 (Howley-Rowe, 1998a-g; 1999a-c), Quest staff observed that some school 
teams attended project events consistently, contributed frequently to network activities, and remained 
in regular contact. Other schools continued their involvement, but with less intensity, while a third 
group of schools chose not to maintain their participation with the network. 

In addition, Quest staff recalled the challenges they had faced when recruiting schools to 
participate in the network. Elementary schools were relatively easy to recruit, whereas high schools 
often reported that they were unable to participate. This led project staff to hypothesize that high 
schools operated in a context inhibiting their ability to become engaged in a long-term endeavor like 
Quest, whereas elementary schools were less hindered in so doing. 

A review of the literature suggested that a variety of factors at the school level and at the 
network or project level impact a school’s capacity to sustain involvement over time in a reform 
effort facilitated by an external agency or consultant. At the school level, logistical factors such as 
lack of time to attend project events or to implement improvement strategies (D’Amico & Corbett, 
1988; Louis & Miles, 1990), difficulty acquiring substitute coverage during teacher attendance at 
events (Selzter & Himley, 1 995), scheduling conflicts (Education Commission of the States, 1 996), 
and lack of funding (Useem et al., 1995) are impediments to ongoing participation. These 
fundamental, practical considerations appear to bear considerably on a school’s ability to become 
and remain involved in an externally facilitated change process. 

Several factors associated with leadership also appear to be important to a school’s continued 
involvement with a reform effort. Turnover in leadership threatens continuity, for instance 
(Education Commission of the States, 1996), as does inconsistent principal participation in project 
efforts (Goldman & Dunlap, 1990). A building leader’s vision (Slavin, 1998) and the district stance 
vis a vis the reform (Cuttance, 1994; D’Amico & Corbett, 1988; Useem et al., 1995) also are found 
to be important factors in sustaining reform momentum. Schoolwide consensus for the initiative is 
likewise noted as a factor significant to the livelihood of reform endeavors (D’Amico & Corbett, 
1988;Honig, 1 994; Louis, Marks, & Kruse, 1994; Louis & Miles, 1990; Slavin, Dolan, & Madden, 
1994). Citing an evaluation of eight schools participating in the Essential Schools network, Honig 
reports that "where reform did not take, no real consensus for change was initially developed; indeed, 
many teachers who gave lip service to the reforms at the start became hostile when changes 
threatened to affect them directly" (1994, p. 794). Thus, schoolwide agreement to undertake change 
impacts personal willingness to tolerate the ambiguity and additional work associated with 
implementation. 

Other issues influencing the success of reform include adequacy of information given school 
staff about the improvement project (Goldman & Dunlap, 1990), relevance of the reform to the 
school (D’Amico & Corbett, 1988; Slavin, 1998), and personal beliefs about education and change 
(Cuttance, 1 994). Some researchers suggest that the subjective experiences and perspectives of those 
involved in reform are vital to its viability (Fullan, 1991); others contend that individuals confront 
particular concerns as they progress through implementation (Hord et al., 1987). The history of 



prior reform efforts within a school is also forwarded as an important variable (D’Amico & Corbett, 
1988). 



School structural issues are additional factors important to the success of school reform. 
Reform at the high school level may confront greater challenges than reform undertaken at 
elementary schools because of "high school organizational complexity" (Useem et al., 1995). Put 
another way, the degree of interdependence between grade-level groups, departments, or teams plays 
a role in how well a reform effort is received and supported over time (D’Amico & Corbett, 1988). 

Some of the literature suggests that the success of school improvement work depends as 
much on characteristics of the particular reform or approach as on school factors. Moreover, the 
ways a reform coincides with a school’s goals, values, and readiness for change significantly impact 
its longevity. D’Amico and Corbett (1988), for instance, argue that any improvement initiative must 
address one of a school district’s two highest priorities if it is to succeed. They elaborate that 
implementation of reform is the result of the interaction between the conditions of local context and 
the processes used to carry out improvement efforts. Similarly, Slavin (1998) contends that change 
may not take place because of a mismatch between the type of reform and a school’s readiness for 
it. 



Nonetheless, Quest staff found relatively little research on the factors that impact schools’ 
willingness to undertake school improvement efforts with the assistance of outside consultants or 
organizations. Nor was there much research concerning the variables influencing schools’ 
involvement over time in such efforts. Other questions of interest were whether respondents from 
elementary and high schools found various factors of more or less importance to their schools’ 
participation, and whether those from schools of varying degrees of involvement with the project 
also made different assessments. 

Quest staff decided to conduct an exploratory, multimethod study of issues the literature 
suggested were significant to schools’ engagement with reform by sampling participants in the Quest 
network for continuous improvement. This report describes the study and its findings. 

The primary audience for this report is Quest staff. It is intended to provide them some 
empirical elaboration of the factors a sample of Quest network participants perceived to be important 
to initial and continued involvement with a long-term school improvement project. Additionally, 
findings from this study may assist Quest staff as they continue to develop the project. Secondary 
audiences for this report include others interested in sustaining school reform initiatives over time, 
from school reform model developers to central office administrators to other school improvement 
researchers. 
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METHODOLOGY 



Using several data sources in order to corroborate theses is what Brewer and Hunter (1989) 
call “multimethod research” or “triangulation.” This approach posits that the strengths of each 
method will compensate for the weaknesses in others, ultimately providing a more complete account 
of that being studied. Therefore, a variety of data was collected throughout this study to provide a 
fuller account of the factors assisting or impeding schools’ participation in the Quest network. 

First, focus group interviews were conducted with participants at each of two rallies 
convened in February 1999. Only those network members who had attended at least one prior Quest 
event were asked to participate in the focus groups; project staff hypothesized that those who were 
attending a Quest event for the first time might have less knowledge about the interview issues. 
There was a total of 147 individuals in the network who had attended at least one project event 
before February. New attendees participated in an orientation session while more veteran members 
were interviewed via focus groups. 

The focus group methodology was chosen because it would allow staff to gather a variety 
of perspectives from a larger number of network members than they might obtain during phone 
interviews. In addition, the presence of network participants at the rallies made the facilitation of 
focus groups all the more efficient. Ultimately, those who participated in the focus group interviews 
comprised a convenience sample of those individuals from moderately to highly engaged schools 
who were present at the February 1999 rallies (N=41). No attempts were made to contact the 
remaining 49 team members from the same schools who did not attend, as project staff thought the 
convenience sample was sufficient for exploratory purposes. 

Three focus groups were conducted between approximately 3:00 and 4:00 p.m. on February 
15,1 999, at the Patrick Henry Hotel in Roanoke, Virginia, during a Quest high school network rally 
that six school teams attended. One focus group consisted of five building administrators, while the 
remaining two groups of six and seven participants included teachers, parents, and students. Each 
focus group was led by a Quest staff member trained in focus group facilitation, who used a 
predesigned interview protocol. 




Three more focus groups were held between approximately 3:20 p.m. and 4:00 p.m. on 
February 22, 1999, at the Wyndham Garden Hotel in Lexington, Kentucky, during a Quest 
elementary school network rally that six school teams attended. As in the earlier round of focus 
groups, one consisted exclusively of five school building administrators. The other two groups, of 
nine each, included teachers and parents. Again, the focus group interviews were conducted by 
Quest staff trained in focus group facilitation using a predesigned interview protocol. A total of 41 
Quest network members were interviewed via the six focus groups conducted in February 1999. 

However, those network members who attended the February rallies tended to hail from 
schools that had been relatively committed to and engaged with the project over time. Quest staff 
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also hoped to understand the perspectives of those who had been less engaged with, or had dropped 
out of, the network (N=57). During the spring and early summer of 1999, therefore, project staff 
conducted telephone interviews with members of recidivist Quest teams and with network 
participants whose schools had not sent teams to the February rallies. Two weeks before phone 
interviewing commenced, project staff mailed all prospective interviewees a one-page letter 
describing the study, requesting their participation, and providing contact information. Twenty-six 
of the 57 individuals in this subgroup participated in the telephone interviews. 

Survey data also were collected from network members, former and present, who had 
attended at least one project event before the February 1999 rallies. Attendees meeting this criterion 
at both rallies were requested to complete the Quest Engagement Survey following the focus groups. 
The survey consisted of 24 diverse factors hypothesized to be important to schools’ involvement in 
a school improvement network such as Quest. These factors were based upon variables suggested 
by the literature as well as several proposed by Quest staff. Respondents were asked to rate on a 5- 
point Likert-type scale how important each factor was, first, to their school’s initial decision to 
become involved with Quest, and second, to their school’s sustained involvement with the network. 
Anchor points on the scale were 1 ( very unimportant) and 5 ( very important), and response options 
included a don ’t know reply. 

The surveys were collected by focus group facilitators following each focus group. Thirty- 
nine surveys were returned from the total 41 focus group interviewees. Phone interviewees from 
recidivist or less engaged schools were also asked to complete the survey. Quest staff offered to 
administer the survey over the phone or to fax the survey for respondents to complete on their own. 
Later in the interviewing cycle, when school was no longer in session, respondents were offered the 
additional options of receiving the survey by mail or e-mail. Nineteen surveys were completed by 
the 26 respondents who participated in the phone interviews. Thus, a total of 58 surveys were 
completed and returned. If the total sample is conceived to be those who participated in the focus 
groups (N=41) or in the telephone interviews (N=26), the survey return rate would be 88%. More 
conservatively, including focus group participants (N=41) and all those Quest staff attempted to 
contact by telephone (N=57), the return rate would be 59%. 

In sum, 67 Quest network members participated in the study, representing 46% of the total 
147 network members. Forty-one participated in focus groups, and 26 in telephone interviews. Of 
the total study participants, 58 completed the survey, 39 of whom were also focus group participants 
and 19 of whom were also phone interviewees. 

Quest staff hypothesized that participants from schools with different levels of involvement 
with the project might confront different issues impacting their participation, as might high school 
and elementary schools. Surveys were coded according to the school level of respondents, and Quest 
staff rated the level of each schools’ involvement in the project on a 3-point scale. Schools assigned 
a 1 had dropped out of the network after participating in one or two early Quest events. Schools 
receiving a rating of 2 were moderately involved in the network, and those rated 3 were highly 
involved. The ratings were based on Quest staff s assessments of how consistently schools attended 
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project meetings, their use of Quest as a resource beyond scheduled project gatherings, their level 
of communication with Quest staff between events, and their apparent willingness to continue 
participation. Three Quest staff conducted the ratings, with a high level of consensus about rating 
assignments. Four schools received a rating of 1 (three elementary and one high school), ten a rating 
of 2 (three elementary and seven high schools), and seven a rating of 3 (five elementary and two high 
schools). Eleven participants from schools rated 1 returned the survey, 22 from schools rated 2, and 
25 from schools rated 3. 

Following data collection, five of the six focus group interviews were transcribed. The sixth 
focus group accidentally was not tape recorded, although the facilitator wrote detailed notes during 
the interview. These were typed for use during data analysis. Quest staff recorded responses received 
during each telephone interview on an interview protocol form. Replies were later entered into a 
WordPerfect file. Focus group data were analyzed by theme using NUD*IST software, and 
telephone interviews were analyzed by question and theme manually. Each theme was coded, and 
the occurrence of each theme was tabulated. Survey data were entered into an SPSS database, and 
statistics were analyzed using the same software. 

Descriptive statistics for the entire sample— elementary and high school respondents 
separately, and respondents from least, moderately, and highly engaged schools separately— were 
generated using SPSS. T tests and analyses of variance (ANOVA) were also conducted, although 
it should be noted that the assumptions of these two statistical tests were violated in this study. The 
sample was not random, nor was it assumed that the data were drawn from a normally distributed 
population or that the samples had homogenous variances. Such tests of statistical significance were 
used in an exploratory fashion, as the purpose for Quest staff of conducting them in this study is to 
identify differences for future, more rigorous exploration. Nonparametric tests (Mann- Whitney U 
and Kruskall-Wallis) were also performed, their results closely approximating those of the 
parametric tests. Post hoc tests following ANOVAs included the Scheffe test for groups determined 
by Levene’s test for homogeneity of variance to have equal variances and Dunnett’s C for those not 
possessing equal variances. 

There were several limitations to this study. First, response rates for participants from 
schools that had dropped out of or were less engaged with the Quest network were lower than those 
from schools that were moderately to highly involved. More engaged schools sent teams to the 
February rallies at which data for this study were collected, enhancing their response rates. Data 
collection from less engaged or uninvolved schools not in attendance at the February rallies was 
hampered by the challenges of phone interviews: Reaching teachers at school was difficult given 
their classroom commitments, some potential interviewees did not return phone calls, and home 
phone numbers were unavailable for 2 1 individuals from less involved or recidivist schools. The 
lack of home phone numbers particularly impacted response rates as much phone interviewing took 
place at the end of the school year. 

A related limitation concerns sampling. Although convenience samples and smaller samples 
are appropriately used for exploratory studies, their validity and generalizability are restricted 
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(Brewer & Hunter, 1989). In other words, although the sample for this study was drawn from the 
population of Quest network participants, and may be generalizable to other Quest members, it may 
not represent the perspectives of educators involved in other reform endeavors because of the use 
of nonprobability sampling techniques. On the other hand, and appropriately for an exploratory 
investigation, the sample was drawn from a theoretically defined universe— school community 
members engaged in school reform work. Thus, as Brewer and Hunter argue, data from this sample 
can be used to generalize by synecdoche as "a claim that the essential features of the larger social 
unit are reproduced in microcosm within the smaller social unit, and that by studying them in micro 
we might make inferences about the macrostructure of which they are a part" (p. 1 23). Nonetheless, 
statistical generalization is compromised in this study. 

In addition, although it might have been revealing to investigate, for instance, differences 
between highly engaged elementary and high schools, or between least engaged elementary and high 
schools, such an examination would have rendered cell sizes too small to have much validity. 
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FINDINGS 



Focus Group Findings 

Network participants in attendance at the February 1999 rallies who had participated in at 
least one prior project event were asked during focus group interviews what factors were most 
important to their school’s initial involvement in the Quest network and to their school’s sustained 
involvement (see Appendix B). Probes on the focus group protocol requested that respondents 
differentiate between school and Quest factors. Analysis of the focus group transcriptions reveals 
that several themes appear important to both elementary and high school respondents. 

By far, the theme most frequently mentioned by interviewees was the centrality of 
administrative support to their initial and sustained involvement in the Quest project. Participants 
reported that often the school building administrator or principal, rather than teachers or other school 
community members, initially noted the relevance of Quest to current school endeavors: Five high 
school and eight elementary school focus group interviewees noted that building administrators had 
directed or suggested involvement with the network. As two focus group interviewees described it, 

"Well, Dr. Baldwin 1 just presented to the site-based council one night when we met saying 
that she had been contacted by AEL and she thought it would be a really good involvement 
for us. At that time, we still had to do the school improvement and all those plans and she 
thought it would help us in writing all those." [elementary school participant] 

"I believe it was our improvement plan and continuously looking for new ways to improve 
our school. And we were asked if we wanted to participate. She didn’t just decide yes, 
we’re going .... The principal did present it as a good opportunity to go and learn some 
new things perhaps, or new strategies to improve our school. And ... it did go along with 
our school improvement plan, also." [elementary school participant] 

Rather than making explicit the relevance of the project to school efforts, central office 
administrators were reported to have simply directed staff to attend the first Quest rally or presented 
Quest as a professional development opportunity of which they might take advantage. Focus group 
interviewees noted 11 times that a central office administrator had facilitated their school’s initial 
participation in the project. 

"It was our superintendent. He just asked me to go as president of student council and he 
figured that he would take me along. He said I could see problems he couldn’t see and that 
maybe we could fix them. Fix problems that he couldn’t." [high school participant] 



'All names used in this report are pseudonyms in order to protect the confidentiality and 
anonymity of those participating in the Quest network. 
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"Our [involvement] was directed from the central office that this was a great program that 
our school should get involved in and it was very important knowledge on the first Quest trip 
by us who were coming as to what Quest was or what it was about and I was really quite 
surprised to find out it was a long commitment because I don’t think we really knew what 
Quest was other than someone in our central office said, ‘This is a good thing. You need to 
go.’" [high school participant] 

"We had an interim superintendent again who was involved with AEL and really showed a 
lot of concern for our system even though he came there as a short-time superintendent, and 
he got us involved." [elementary school participant] 

"Mr. Young— I think he made the decision and told faculty who he wants to be involved . . 
..Sol think it was more or less the principal’s decision." [elementary school participant] 

Similarly, both elementary and high school participants reported that continued and active 
central office and building administrator support for involvement in the Quest project was vital to 
their school’s ongoing participation. The importance of central office support was noted four times 
by elementary and four times by high school interviewees. Overwhelmingly, however, respondents 
thought that the support of the building-level administrator was most fundamental to their school’s 
continued involvement with Quest. Mention of this theme was made 1 1 times by elementary 
respondents and 1 9 times by their high school counterparts. Participants put their perspectives in 
these ways: 

"I feel like right now [our involvement with Quest] is really high because our principal is 
really actively involved, so therefore if a principal is actively involved and you know you 
have your school that’s going to follow." [elementary school participant] 

"I think the people at the administrative level are absolutely critical, because a student or a 
parent or even a teacher probably can’t make it go once it gets back home. And so I know 
we need our administrative person, and we’ve got one with a lot of energy now, and I think 
it’s going to make a huge difference." [high school participant] 

"I think the teacher can keep it alive but maybe couldn’t have made it really push. But 
having a committed administrator there is just really important." [high school participant] 

"If you don’t have the administrator, you have nothing at all." [high school participant] 

Seven of the comments suggesting the significance of building-level administrative support 
for sustained involvement in the Quest network also referred to such administrators’ willingness to 
secure or broker funding, released time, and substitute coverage to enable participant attendance at 
project meetings. For instance, 
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"I think what makes it easy for us to be involved is that our principal is totally in agreement 
and dedicated to the situation, and so that’s never a question as far as freeing time to attend 
meetings or expenses or whatever it takes. She thinks it’s a good thing, and we are 
committed to do this." [elementary school participant] 

"I think it really has a lot to do with our staff and the fact that they can find money. Our 
principal can find money in places that we have no clue. We do not ask. We’re just glad the 
money’s there and she pays for us to come and it really helps. Because if we had to pay for 
it out of our pockets, I really don’t think as many people would be able to come." 
[elementary school participant] 

"Again, I think resourcefulness [is important to involvement]. I think, you know, having the 
principal back this really makes things possible. Because as I see it, and I’m on the outside, 
it costs the school money to be involved and all the teachers who are here . . . had to find 
money to find substitutes that are there with our children today and tomorrow. And just 
being here ... so, to stay at the hotel and the food and everything ..." [elementary school 
participant] 

"While we can’t get [our principal] to come [to Quest events], he’s willing for us to go, and 
he’s very supportive, and he helps us try to find funding and substitutes and all those kind 
of things." [high school participant] 

Eleven comments also indicated the necessity of schoolwide consensus for involvement in 
Quest, including teachers, parents, and students, to the success of continued participation and to 
implementation efforts. Three participants reported that their school communities supported their 
affiliation with Quest, which in turn enhanced their schools’ ongoing participation in the network. 
Asked what helped to sustain his school’s involvement in the project, one elementary school 
participant replied, "Probably the active participation from all the faculty and staff and parents that 
we have there along with the principal." The remainder reported less schoolwide agreement about 
involvement, which they saw as impeding the success of their Quest work. As one elementary 
network interviewee phrased it, "If teachers have not really been involved, it’s been hard for them 
to understand what we’re doing." 

A total of nine comments were offered about the importance of funding to participation in 
a network such as Quest. One high school focus group interviewee specifically noted the importance 
of funding received from AEL to support continued participation in the project: "The other factor 
is the money provided by AEL. If we didn’t have the money, I wouldn’t be here." Similarly, one 
participant noted that a reform model implemented at her high school provided financial assistance: 
"That’s something that our High Schools That Work team has that we [Quest team] do not have . . 
.. They’ve got the money from the High Schools That Work that’s dedicated . . . just to the sub 
days and planning days." 
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Fifteen comments indicated that, while administrative support for attendance at Quest events 
was important, at least equally important was active administrative support for the school 
improvement endeavors Quest team members learned of during project events and intended to 
implement at their schools. Three high school interviewees, for instance, reported: 

"[The administrators are] all for us coming down and coming back with new ideas and 
putting it to work at the school, and when we get back it’s pretty much left up to us to do all 
the work." 

"I think we’ve got an overall desire to improve, yet I’m not so sure that’s driven by the 
administrators. It’s that we, as Quest members, continue to go in and say, ‘Hey, what are we 
doing on Quest lately?’ or something like that. We kind of prod her along. Not that she’s 
against Quest or anything like that. She just doesn’t have the time for it." 

"And I think that’s a blind spot for administrators, because they send staff to meetings like 
this and expect us to come back and do things, and they don’t realize the position that puts 
us in since we don’t have the authority. My administrator has said to me, ‘Well, why didn’t 
you take that and run?’ I couldn’t run. How?" 

Similarly, an elementary school participant reported, 

"Our principal does her very best to motivate and inform and implement as many things as 
possible, and the teachers that have been to the Quest rallies, we’ve gone back and tried to 
share with colleagues and then implement things that we have chosen or found most valuable 
to use in our classroom." [elementary school participant] 

Among the themes most often mentioned by focus group participants was the relevance of 
Quest goals to school goals, or the ways Quest coincided with particular school efforts. Participants 
reported that such relevance was necessary both to their schools’ initial and continued engagement 
with the network. This theme was mentioned seven times by high school interviewees and 14 times 
by their elementary school counterparts. One respondent thought the relevance of Quest to school 
undertakings indicated redundancy: 

"It’s been a reoccurring [sic]— I don’t want to say it’s a problem. It’s an occurrence that 
because we have High Schools That Work and the School Improvement Plan and Quest, we 
have school improvement going on in the school and it seems like the same things that we’re 
doing for this we can also include in doing this in the other projects that we have going on 
at the time." [high school participant] 

But other focus group participants who spoke about how school and Quest goals overlapped 
assessed such relevance positively, noting its significance to their involvement with the project. 
Several quotes are illustrative of this point. 
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"I would say that through Quest we have been able to use a lot of their ideals to enhance ours 
and to assess ours to see if we’re going in the right direction and then I think as Sally said, 
too, I mean they overlapped and it’s just like a support system so you’re able to accomplish 
two things at once while you’re doing whatever type of school improvement you’re doing." 
[high school participant] 

"Occasion had something to do with it, but then also we were at the process— just starting the 
process of accreditation. Getting re-accredited in School Renewal Process, so we used Quest 
as a vehicle to help out with that." [high school participant] 

"When I think about our school . . . we’re constantly trying to improve ourselves and this is 
just an outlet in order to do that. I mean, it’s the best one I think that we’ve found to help 
us continuously improve ourselves because we all want to be the best we can be and AEL 
helps us do that." [elementary school participant] 

"I think that it was our parent program and it just kind of fell into what we were already 
doing instead of something new that we were ... it was just kind of ... we were already 
doing it and we could just continue." [elementary school participant] 

Another theme mentioned often was the ambiguity participants felt as they began their 
involvement with Quest. Twelve comments were made about this theme, although it should be 
noted that 10 were forwarded during one high school focus group in which interviewees had a 
spontaneous discussion on the topic. Eight of the comments indicated that participants felt the 
information they received initially about Quest was unclear. 

"I was really quite surprised to find out it was a long commitment, because I don’t think we 
really knew what Quest was other than someone in our central office said, "This is a good 
thing you need to go to." [high school participant] 

"I knew it was about school improvement, and that was about it." [high school participant] 

The remaining four of these comments noted continuing challenges Quest team members 
faced in communicating about Quest to their school colleagues. For example, 

"And that’s partly our responsibility and partly AEL’s for not giving them that [information] 
initially, I guess. That if they knew more about it, and I think especially if they knew more 
about that it’s not just a bunch of research, but it’s actually ideas being shared and that kind 
of stuff, I think there’d be more teachers involved." [high school participant] 

"But the paper you gave us today, the two pages on Quest, we should have had day one, 
because it’s been unbelievably difficult to explain to parents, to explain to students, to my 
student government . . . you know, what this is, because there hasn’t been anything that says, 
you know, what it is." [high school participant] 
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Mention was made nine times of the difficulty in managing competing priorities in order to 
continue participation in the Quest network. These priorities ranged from implementation of various 
reform initiatives to regular school duties such as teaching and grading. 

"We have staff development that we have to do and school improvement that we have to do 
in Tennessee, and sometimes the feeling is this is in conflict." [elementary school participant] 

"What would get in the way of our involvement would be the programs that we have still 
require a lot of time. For example, our after-school program." [elementary school 
participant] 

"We had difficulty trying to find other teachers that would come with us because we had just 
went to the block [schedule] this year, and no one wanted to leave their class for two days." 
[high school participant] 

A related concern was the time away from school that attendance at Quest events required. 
Mention was made of this theme six times by focus group respondents. Participants reported that 
their attendance depended on, as one put it, a "sacrifice" in both professional and personal time. A 
high school administrator said that taking time to attend Quest events created a rift in morale: "It 
becomes a morale issue for those teachers who are left behind to deal with things in our absence. 
When we get back there will be morale problems. ‘If you hadn’t gone to Quest . . .’" 

Time to reflect on their progress, formulate plans, and implement improvement efforts was 
another theme of some importance to seven focus group respondents. Participants reported that 
while attendance at Quest meetings was affirming and useful, the time needed to implement school 
improvement strategies learned of from Quest was sometimes an impediment to their continued 
involvement. Asked what factors hindered school participation in Quest, one elementary school 
respondent put it this way: 

"Time within your school day for faculty to get together and talk about issues and do real 
collegial sharing and discussion of the pros and cons of a particular program or something 
that Quest may have brought up and that you really need the faculty as a whole to understand 
because there are just two or three of us maybe that saw the Interview Design. Now we need 
to take it back to our faculty .... When you have a school day that is extremely long and 
complex and there’s no time for the whole faculty to get together except once a month after 
school. So I think that the time is a critical factor." 

Eight comments were made about the networking with colleagues facilitated by Quest 
participation. Four participants noted that the potential of such opportunities to meet and exchange 
ideas with other educators informed their schools’ first decision to become involved in the project. 
"I saw it as a wonderful opportunity to engage . . . with a larger group and to get ideas from other 
schools," reported one elementary network member. Four other respondents noted that the 
networking opportunities helped to sustain their engagement in the project. As one such elementary 
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school participant put it, "It’s kind of give[n] us confidence to know that other schools are wanting 
all over the areas, wanting to improve our schools, and it’s just give[n] us some ideas hearing from 
the other schools." 

Seven elementary school participants reported that prior or current involvement with AEL 
facilitated their initial or continued participation with the Quest project. Five noted that the quality 
of their earlier experiences with AEL services convinced them to join Quest. 

"I think we’re involved because past work with AEL has been so successful and knowing the 
people and the quality of work . . . when there was an opportunity to do something else we 
jumped at it." 

"We got involved because we’d had such great experiences with AEL and knew that 
anything AEL wanted to do was going to be top drawer because of the quality of the people 
and the sophistication and professional integrity of the organization." 

Two others mentioned their involvement with another program facilitated by Quest staff, 
QUILT (Questioning and Understanding to Improve Learning and Thinking). However, their 
comments do not clearly elaborate the relationship between participation in QUILT and Quest. 
Asked to describe their level of involvement with Quest, they replied, 

"We’ve done the QUILT program and that has kept us abreast of everything that’s going on, 
and we have had staff developments on QUILT .... So, we are involved, very involved." 

"One of the people . . . involved in the development of the QUILT program, is an 
administrator at our school .... Certain ones are going to be going for the QUILT training 
and then coming back and training our staff as professional development this summer and 
to implement that into our daily working with the higher level questioning. And so I’d say 
we were very much involved." 

These comments might be interpreted in at least two ways. First, perhaps the respondents 
meant to suggest that their additional contact with Quest staff through QUILT enhanced their 
commitment to or knowledge about Quest. Or, maybe the participants had elided the two projects 
conceptually. 

Two elementary and five high school focus group interviewees reported the importance of 
scheduling to their continued participation in Quest, six citing their inability to attend events at 
particular points throughout the school year, such as during the administration of final exams. A 
seventh participant, from the high school network, spoke of the timing of project gatherings as 
contributing to momentum: "The timing of the conferences has been important, because as soon as 
you kind of lapse back into that dull lull, it’s almost like there’s another conference, and you renew 
your energy." 
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Yet another theme of some significance concerned a school community’s willingness to 
accept feedback and implement change. Six comments were offered to suggest that involvement in 
Quest required a commitment to explore avenues of growth, and that for those interested in 
continuous improvement, change rendered with the support of Quest helped sustain school 
engagement in the network. Asked to describe what factors helped her school remain involved with 
the project, one elementary network participant responded, 

"I would say a willingness to accept change, because when you’re committed to Quest, 
things are going to happen. And if you are reluctant to change or drag your feet this is not 
for you. But if you’re willing to change ... and when you’re ready to change, then it’s going 
to be an alive environment, a moving, good thing." 

Likewise, a high school network respondent reported, 

"You wouldn’t participate in this, I don’t think, unless you were willing to acknowledge any 
fault that you might have. You know, be willing to make improvements. I don’t think that 
you would participate in this as a school if you weren’t willing to do that to some degree." 

Comments made by three elementary and three high school network focus group participants 
intimate the ways affirmation received through Quest promotes ongoing participation in the project. 
Three participants spoke of affirmation in terms of "renewal" and "revitalization." The three 
remaining comments discuss affirmation in terms of using Quest staff as a sounding board for ideas 
or as "critical friends." For example, an elementary participant said, 

"One of the things that I’ve found helps us stay involved is that the AEL staff has been so 
supportive and affirming. The co-venture was great, and to have people with the expertise 
and the experience of the AEL people saying that what we’re doing is right and it’s good and 
it’s unique . . 

Six comments, four from elementary and two from high school network members, were 
offered concerning the logistical difficulties to participation in Quest faced by small schools. 
Practical issues included lack of resources to support participation, challenges in "fielding a team," 
and the burden of maintaining many projects with few staff. 

Other themes were forwarded by focus group participants with relatively less frequency. For 
instance, five participants discussed the importance of Quest as a resource for research and strategies 
to their schools’ participation in the network, and four of their personal motivation to attend project 
events. The ways Quest encourages development of a Quest school team contributed to their 
schools’ sustained participation, according to four respondents. Three comments suggested that the 
overlap of school and Quest philosophy or vision contributed to involvement; three others cited 
previous positive reform efforts. 
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Themes mentioned only twice included the importance of personal relationships with Quest 
staff, the inclusiveness of the Quest network, personal availability to attend project meetings, and 
the ways Quest participation intersected with personal growth. 

Table 1 summarizes the themes most frequently mentioned by focus group participants as 
important to their schools’ initial and sustained involvement in Quest. 



Table 1 

Important Factors Influencing Quest Involvement: Summary of Most Frequently 

Mentioned Focus Group Themes 



Theme 


Frequency 


Building-level administrative support for sustained involvement 


30 


Relevance of Quest to the school 


21 


Active building-level administrative support for school improvement projects 


15 


Building-level administrative support for initial involvement 


13 


Ambiguity about Quest 


12 


Central office support for initial involvement in Quest 


11 


Schoolwide consensus for sustained Quest involvement 


11 


Funding for sustained involvement in Quest 


9 


Other priorities competing with sustained Quest involvement 


9 



Interview Findings 

Twenty-six individuals from four low and five moderately engaged Quest schools 
participated in telephone interviews conducted by trained Quest staff. Respondents were asked via 
a structured interview protocol (see Appendix C) to describe what factors at their school and of 
Quest had contributed to their schools’ initial involvement in the network. Respondents from 
schools that had left the Quest network were requested to describe what school and Quest factors had 
hindered their schools’ continued participation in Quest, while schools maintaining their 
involvement in the project were asked to describe what school and Quest factors had helped and 
hindered their ongoing participation. 

Eight of the 26 interview respondents provided replies consisting of two or more themes 
when asked what school factors had been most important in determining their schools’ initial 
participation in Quest. The remainder offered replies containing one theme. Eight respondents 
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reported that their schools’ initial involvement had been most influenced by the interest in exploring 
the potential relevance of Quest to their schools. "We are a progressive school always looking for 
new programs," said one participant. Another respondent noted that her large school had been 
"trying to look at ways of improving things for the entire school population." Seven respondents 
indicated that the relevance of Quest to their schools had been apparent and influenced their decision 
to become involved. Of these, six noted specific areas to which they hoped Quest would be relevant: 
Two mentioned parent involvement, one community involvement, one work with small schools, one 
consolidation issues, and one concern with test scores. 

Five responses indicated that the principal had initiated the schools’ initial participation in 
Quest. Four respondents reported that initially they had very little information about the project. 
Initial involvement was facilitated by the district office, according to the three respondents from one 
school. Two respondents reported that they did not know what school factors had impacted their 
schools’ first association with Quest. Two others said prior experience with AEL had influenced 
their initial decision. One reply each suggested that networking opportunities, faculty willingness 
to participate, availability of funding to attend, and curiosity had been important factors impacting 
schools’ initial involvement in Quest. One response was unclear and was not coded. 

Eleven of the 26 interview respondents offered replies with multiple themes when requested 
to describe what characteristics or factors of Quest had been most important in determining their 
schools’ initial involvement in the network. The remainder offered replies with one theme. Nine 
respondents indicated that they had little or no information about Quest as their schools became 
involved and therefore could not offer definitive analyses of what about Quest encouraged their 
schools’ participation. (Nonetheless, four of these respondents attempted to guess what Quest 
characteristics had been important.) 

Five responses indicated that their schools’ first involvement with Quest had been a trial run, 
and all five reported that their schools had assessed their experiences positively. Three of these, 
however, added that they had encountered "nothing new," as one put it, at their schools’ first Quest 
rally. The relevance of Quest to their schools was noted by four participants as significant to their 
schools’ first association with the project. Three replies each suggested the importance of 
networking opportunities, prior knowledge of AEL’s services, and the opportunity to explore what 
Quest had to offer. Two respondents replied that Quest’s non-directive philosophy had been 
important to their schools’ initial engagement. One such respondent reported that she "liked the 
‘design-your-own’ aspect" of Quest. Two responses each indicated the significance of the 
principals’ initiation of involvement and of the opportunities for acquiring new information through 
Quest. One respondent suggested Quest processes had been important, while another noted the 
importance of opportunities for team building offered by the network. One reply was unclear and 
remained uncoded. 

Nineteen of the 26 respondents provided replies with multiple themes when asked to identify 
what school factors had hindered or helped their schools’ sustained involvement in Quest. More, 
and more lengthy, responses were offered concerning impediments to schools’ continued 
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participation than were offered concerning factors facilitating long-term involvement. In other 
words, respondents had more to report about impediments than about encouragement of their 
schools’ involvement. 

In terms of hindrances to ongoing participation in Quest, 12 replies mentioned the presence 
of competing priorities at their schools. As one participant said, "[We’re] completely 
overcommitted. The county has grant money with which they’ve started summer academics, and 
weekend and evening programs. [We have] no time at all. [There is] a shortage of people to do the 
work; many jobs have been combined." Similarly, a principal reported, "[I] am overwhelmed when 
I get back from meetings. [There are] so many things in a high school that require my attention . . 

. . I work 10-14 hours per day." One respondent listed the many programs at her school for which 
staff had assumed responsibility: "Lots of things, federal grants . . . before-school program, after- 
school program, tutoring program . . . CCLC grant . . . summer academy . . . community classes. . 

. . [We’re] just burning out." 

Eight replies indicated that a lack of administrative support impeded their schools’ continued 
involvement in Quest. Two of these mentioned lack of central office support; the remainder 
pertained to the paucity of building-level administrative support. Six responses indicated that a lack 
of funding to attend Quest functions hindered their continued participation; four of these responses 
also contained the theme concerning lack of administrative support. One such participant said, 
"Money. We’d keep going if we didn’t have to beg for it from the principal." Six replies suggested 
that a lack of time discouraged their schools’ involvement. Four responses each indicated that a 
particularly difficult school year and a faculty reluctant to participate in Quest had hindered their 
schools’ sustained association with the project. Two respondents reported that difficulties in 
acquiring substitute coverage for classes while they attended network gatherings had inhibited their 
schools’ participation. School communication problems, lack ofparent and community involvement 
in Quest, and faculty confusion about the relevance of Quest to their school were identified by one 
respondent each as impediments. One respondent reported that nothing at her school had hindered 
their involvement in the network. 

Two replies each suggested that principal support and the relevance of Quest to the school 
had been important to their schools’ ongoing participation. The adequacy of Quest communications, 
sharing ideas, and dedication of Quest team members were cited by one participant each as 
encouraging their schools’ involvement. 

Again, when asked what characteristics or factors of Quest inhibited or encouraged their 
schools’ continued participation, more responses were offered concerning impediments. 
(Nonetheless, nine respondents indicated that nothing about Quest itself had hindered their 
participation, although three of these went on to mention that attendance at project events did require 
funding, which was a concern. A fourth participant reporting that nothing about Quest impeded 
involvement later noted that the relevance of the network to her school was ambiguous.) Seven 
respondents each cited competing priorities, lack of time, and funding as hindrances to their schools’ 
continued involvement. Three replies suggested that the relevance of Quest to their schools was 
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unclear. Similarly, one respondent said, "[My school is] just not the type of school that’s going to 
be involved." Two participants reported that they felt a sense of disjuncture after having missed one 
Quest rally: "If you missed a meeting, [it] was hard to go back. [You] felt you had missed out," as 
one phrased it. The other suggested that Quest meetings focus on one theme per year. Traveling 
long distances to attend project meetings was an impediment to two respondents. Reluctant faculty 
and scheduling of events were hindrances to one respondent each. It should be noted that not all of 
the replies address characteristics of Quest itself, instead referring to school issues. 

Eleven respondents reported that characteristics of Quest had encouraged their schools’ 
ongoing involvement in the network, one of whom provided an answer containing two themes. 
Seven responses indicated generically that Quest was "beneficial" or "wonderful," for instance. Two 
responses each suggested that the informative and frequent communications from Quest staff and 
the value of Quest as an information resource supported their schools’ sustained participation. One 
reply noted that the project’s focus on parent and community involvement had encouraged her 
schools’ continued participation. 



Survey Findings 

Study participants were asked to complete the Quest Engagement Survey (see Appendix D) 
on which they were to rate the level of importance of 24 variables to, first, their schools’ initial 
involvement with Quest, and second, their schools’ sustained involvement with the project. Ratings 
for initial and sustained involvement were treated as separate variables, for a total of 48 variables. 
Respondents were asked to rate the importance of each factor using a 5-point Likert-type scale. 
Anchor points on the scale were 1 {very unimportant) and 5 ( very important), and response options 
included a don 't know reply. 

Fifty-eight surveys were completed and returned. Elementary school respondents comprised 
33 of the total and high school respondents 25. Eleven respondents hailed from schools categorized 
as least engaged, 22 from moderately engaged schools, and 25 from highly engaged schools. 

Internal consistency reliabilities of the variables concerning initial and sustained involvement 
were calculated separately using Cronbach’s alpha. With an alpha coefficient of .94, the variables 
addressing initial involvement of schools in the Quest network have sufficient internal consistency 
reliability. Likewise, with an alpha coefficient of .89, the variables concerning schools’ sustained 
participation in Quest also possess satisfactory internal consistency reliability. 



Descriptive Statistics 

Mean ratings for all the factors were calculated; the five most highly rated variables are 
reported here (see Table 2). The variable rated most important to study participants’ schools’ initial 
involvement in Quest was personal beliefs about education and change, with a mean of 4.41 (SD .86) 
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on the 5-point scale. Also ranked as important to schools’ first decision to participate in the project 
were the vision for learning in the school (4.22, SD 1 .09), level of communication within the school 
(3.96, SD 1 .03), relevance of Quest to the school (3.94, SD 1.10), and the match between school and 
Quest network goals. 



Table 2 

Factors Rated Most Important to Quest Schools’ Initial Participation 



Factor 


N 


Mean 


SD 


Personal beliefs about education and change 


54 


4.41 


.86 


Vision for learning in the school 


54 


4.22 


1.09 


Level of communication within the school 


52 


3.96 


1.03 


Relevance of Quest to the school 


50 


3.94 


1.10 


Match between school and Quest network goals 


46 


3.93 


1.08 



While personal beliefs about education and change were also rated as most important to 
schools’ sustained participation in Quest, with a mean of 4.60 (SD .67), there were several 
differences between respondents’ ratings of initial and sustained involvement (see Table 3). 
Relevance of Quest to the school was ranked the second most important factor to sustained 
participation (4.43, SD .80), with the ways Quest supports school improvement (through rallies, co- 
ventures, and symposia, for instance) ranked third (4.38, SD .81). Also highly rated in terms of their 
significance to ongoing involvement with Quest were the availability of funding to attend project 
events (4.33, SD 1.06) and the match between school and Quest network goals (4.31, SD .85). It is 
interesting to note that the variables rated most important to sustained involvement had slightly 
higher mean scores than the variables rated most important to schools’ initial involvement. This may 
be because schools’ initial decision to join Quest required fewer commitments at first, whereas 
sustained involvement required ongoing commitment and resources, perhaps more challenging to 
maintain. In addition, standard deviations were also somewhat smaller for variables pertaining to 
sustained involvement, suggesting there was a bit more agreement among respondents about the 
importance of various factors to their schools’ ongoing participation in Quest. 

Mean ratings were calculated for elementary and high school network participants separately. 
Most important to elementary schools’ initial engagement with Quest was the vision for learning in 
the schools, with a mean of 4.57 (SD .68) (see Table 4). Also highly important to elementary 
schools’ initial involvement were personal beliefs about education and change (4.33, SD .88), the 
amount of time needed to do improvement work at the school (4.16, SD 1.04), the level of 
communication within the school (4.10, SD .92), and the match between school and Quest network 
beliefs (4.04, SD 1.06). 
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Table 3 

Factors Rated Most Important to Quest Schools’ Sustained Participation 
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Factor 


N 


Mean 


SD 


Personal beliefs about education and change 


58 


4.60 


.67 


Relevance of Quest to the school . 


53 


4.43 


.80 


Ways Quest supports school improvement (rallies, co-ventures, etc.) 


58 


4.38 


.81 


Availability of funding to attend Quest events 


54 


4.33 


1.06 


Match between school and Quest network goals 


52 


4.31 


.85 



Table 4 

Factors Rated Most Important to Elementary Schools’ Initial Participation 



Factor 


N 


Mean 


SD 


Vision for learning in the school 


30 


4.57 


.68 


Personal beliefs about education and change 


30 


4.33 


.88 


Amount of time to do improvement work at school 


31 


4.16 


1.04 


Level of communication within school 


30 


4.10 


.92 


Match between school and Quest network beliefs 


25 


4.04 


1.06 



High school respondents rated the importance of the factors somewhat differently than their 
elementary school counterparts (see Table 5). Personal beliefs about education and change were 
ranked most highly, with a mean of 4.50 (SD .83). With a mean of 3.90 (SD 1.04), the match 
between school and Quest network goals was rated the second most important variable to high 
schools’ initial involvement in Quest. Also ranked most highly by high school respondents were the 
amount of time needed to attend Quest events (3.83, SD 1.05), the relevance of Quest to the school 
(3.83, SD 1.11), and the match between school and Quest network beliefs (3.81, SD 1.21). 

Elementary respondents appear to have rated the importance of items to their schools’ initial 
participation in Quest more highly than did high school respondents. In addition, three of the five 
variables rated most highly by high school respondents concern the intersection of school and Quest 
beliefs and goals. In other words, the match between the school and the reform appears to have been 
more important to high schools’ initial participation than to elementary schools’. 
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Table 5 

Factors Rated Most Important to High Schools’ Initial Participation 
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Factor 


N 


Mean 


SD 


Personal beliefs about education and change 


24 


4.50 


.83 


Match between school and Quest network goals 


21 


3.90 


1.04 


Amount of time needed to attend Quest events 


24 


3.83 


1.05 


Relevance of Quest to the school 


23 


3.83 


1.11 


Match between school and Quest network beliefs 


21 


3.81 


1.21 



As with the overall rating of factors, elementary school participants ranked personal beliefs 
about education and change as most important to their schools’ sustained engagement with the Quest 
network, with a mean of 4.61 (SD .61) (see Table 6). But the ways in which Quest supports 
continuous improvement, such as through rallies, co-ventures, and symposia, was rated identically, 
with a mean of 4.61 (SD .61). Also rated most highly were the relevance of Quest to the school 
(4.45, SD .91), the vision for learning in the school (4.45, SD .97), and the match between school 
and Quest network beliefs (4.39, SD .74). Too, standard deviations were below 1.00, indicating a 
relatively consistent level of agreement among elementary school respondents regarding the 
importance of the most highly rated factors. 



Table 6 

Factors Rated Most Important to Elementary Schools’ Sustained Participation 



Factor 


N 


Mean 


SD 


Personal beliefs about education and change 


33 


4.61 


.61 


Ways Quest supports school improvement (rallies, co-ventures, etc.) 


33 


4.61 


.61 


Relevance of Quest to the school 


29 


4.45 


.91 


Vision for learning in the school 


33 


4.45 


.97 


Match between school and Quest network beliefs 


28 


4.39 


.74 
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High school respondents also rated most highly the importance of personal beliefs about 
education and change to their schools’ ongoing participation in Quest (4.60, SD .76) (see Table 7). 
The relevance of Quest to the school (4.42, SD .65) and Quest processes (4.26, SD .75), such as Data 
in a Day and the Protocol process, also received high ratings. In addition, high school respondents 
ranked the amount of information they had received about Quest (4.25, SD .68) and the availability 
of funding to attend Quest events (4.24, SD 1 .09) as among the five most important variables to their 
schools’ continued involvement in the project. 

Table 7 

Factors Rated Most Important to High Schools’ Sustained Participation 



Factor 


N 


Mean 


SD 


Personal beliefs about education and change 


25 


4.60 


.76 


Relevance of Quest to the school 


24 


4.42 


.65 


Quest processes (Data in a Day, Protocol process, etc.) 


23 


4.26 


.75 


Amount of information you received about the Quest network 


24 


4.25 


.68 


Availability of funding to attend Quest events 


21 


4.24 


1.09 



Because Quest staff hypothesized that schools with various degrees of involvement in the 
project might confront different impediments to or furtherances of their initial and sustained 
participation, comparisons of mean ratings were made between participants from least, moderately, 
and highly engaged schools. 

In terms of initial involvement in Quest, most highly rated by schools that had dropped out 
of the network was the vision for learning in the school, with a mean of 4.00 and a standard deviation 
of 1.33 (see Table 8). Also important were the amount of time to do improvement work at the 
school (3.91, SD 1.22) and the amount of time needed to attend Quest events (3.9 1,SD 1.30). With 
a mean of 3.82 (SD 1.17), personal beliefs about education and change were among the top five most 
important factors to least engaged schools’ initial involvement in Quest, as was the relevance of 
Quest to the school (3.70, SD .95). 

Ranked most important to moderately engaged schools’s initial participation were personal 
beliefs about education and change (4.43, SD .81) (see Table 9). Also important were the vision for 
learning in the school (4.19, SD .93), the level of communication within the school (3.95, SD 1.02), 
and the vision of building-level leaders (3.90, SD .97). The factor rated fifth most important to 
moderately engaged schools’ initial involvement in Quest was the amount of time available to 
individuals for professional development (3.85, SD 1 .04). The large standard deviations for the three 
most highly rated factors suggest, however, that there was some disagreement among respondents 
concerning the importance of these items to their schools’ initial participation in Quest. 
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Table 8 

Factors Rated Most Important to Least Engaged Quest Schools’ Initial Participation 



24 



Factor 


N 


Mean 


SD 


Vision for learning in the school 


10 


4.00 


1.33 


Amount of time to do improvement work at school 


11 


3.91 


1.22 


Amount of time needed to attend Quest events 


11 


3.91 


1.30 


Personal beliefs about education and change 


11 


3.82 


1.17 


Relevance of Quest to the school 


10 


3.70 


.95 



Table 9 

Factors Rated Most Important to Moderately Engaged Quest Schools’ Initial Participation 



Factor 


N 


Mean 


SD 


Personal beliefs about education and change 


21 


4.43 


.81 


Vision for learning in the school 


21 


4.19 


.93 


Level of communication within your school 


21 


3.95 


1.02 


Vision of building-level leaders 


20 


3.90 


.97 


Amount of time available to individuals for professional 
development 


20 


3.85 


1.04 



Similar to respondents from moderately engaged schools, those from highly engaged schools 
ranked personal beliefs about education and change as most important to their schools’ initial 
involvement in the Quest network, with a mean of 4.68 (SD .57) (see Table 10). Participants from 
highly engaged schools also rated highly the importance of the match between school and Quest 
network goals (4.50, SD .71) and the vision for learning in the school (4.35, SD 1.15). The match 
between school and Quest network beliefs (4.33, SD 1 .03) and the relevance of Quest to the school 
(4.23, SD 1.15) were rated as the fourth and fifth most significant issues to ongoing participation. 

One important difference between respondents from least and highly engaged schools’ 
ratings of most important factors to their schools’ initial participation in Quest is that three of the 
five top-rated factors to highly engaged schools concern the intersection of the schools’ goals and 
philosophy with Quest’s. Differently, respondents from the least engaged schools rated most highly 
two factors that concern logistical issues at the school level, although the relevance of Quest to the 
school was rated the fifth most important variable to such schools’ initial involvement. 
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Table 10 

Factors Rated Most Important to Highly Engaged Quest Schools’ Initial Participation 
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Factor 


N 


Mean 


SD 


Personal beliefs about education and change 


22 


4.68 


.57 


Match between school and Quest network goals 


18 


4.50 


.71 


Vision for learning in the school 


23 


4.35 


1.15 


Match between school and Quest network beliefs 


18 


4.33 


1.03 


Relevance of Quest to the school 


22 


4.23 


1.15 



An interesting correspondence between ratings from the three levels of school engagement 
is that the vision for learning in the school and personal beliefs about education and change were 
ranked among the top five most important variables to initial participation in Quest by respondents 
from all three levels. 

Comparisons between schools at various levels of engagement were also made of ratings of 
factors to schools’ sustained involvement in Quest. Examination of the tables reveals that 
respondents from all three levels of school engagement rated the importance of factors more highly 
when asked about their relevance to sustained participation than to initial participation. 

Respondents from least engaged schools rated most highly the significance of personal 
beliefs about education and change (4.36, SD .81) and the amount of time to do improvement work 
at school (4.36, SD 1 .03) to their schools’ sustained participation in Quest (see Table 1 1 ). Also rated 
as important were the ways Quest supports school improvement (thrott^ rabies, co-ventures, and 
symposia, for instance) (4.27, SD .90) and the amount of time needed to attend Quest events (4.27, 
SD 1.19). The availability of funding to attend project events (4.09, SD 1.64) was highly rated, 
although the very large standard deviation suggests that there was some disagreement among 
respondents concerning this. As these schools had left the Quest network at the time they were 
surveyed, it can be assumed that these issues were likely impediments to continued involvement. 

Participants from moderately engaged schools rated most important to their schools’ 
sustained involvement in Quest personal beliefs about education and change (4.55, SD .74) (see 
Table 12). With a mean of 4.36 (SD .66), the vision for learning in the school was also important, 
as was the vision of building-level leaders (4.32, SD .57). Respondents from moderately involved 
schools also rated highly the relevance of Quest to their school (4.32, SD .67) and the amount of 
time needed to attend Quest meetings (4.27, SD .77). 
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Table 1 1 

Factors Rated Most Important to Least Engaged Quest Schools’ Sustained Participation 
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Factor 


N 


Mean 


SD 


Personal beliefs about education and change 


11 


4.36 


.81 


Amount of time to do improvement work at school 


11 


4.36 


1.03 


Ways Quest supports school improvement (rallies, co-ventures, etc.) 


11 


4.27 


.90 


Amount of time needed to attend Quest events 


11 


4.27 


1.19 


Availability of funding to attend 


11 


4.09 


1.64 



Table 12 

Factors Rated Most Important to Moderately Engaged Quest Schools’ Sustained Participation 



Factor 


N 


Mean 


SD 


Personal beliefs about education and change 


22 


4.55 


.74 


Vision for learning in the school 


22 


4.36 


.66 


Vision of building-level leaders 


22 


4.32 


.57 


Relevance of Quest to the school 


19 


4.32 


.67 


Amount of time needed to attend Quest events 


22 


4.27 


.77 



Respondents from highly engaged schools ranked the ways Quest supports school 
improvement, through events such as rallies and co-ventures, as most significant to their schools’ 
sustained participation (4.80, SD .41) (see Table 13). The relevance of Quest to the school (4.79, 
SD .41), personal beliefs about education and change (4.76, SD .52), and the match between school 
and Quest network goals (4.70, SD .56) were also deemed most important by respondents from 
highly involved schools. Rated fifth most important were Quest processes, such as Data in a Day and 
the Protocol process, with a mean of 4.68 (SD .48). Also of interest are the small standard 
deviations, indicating less variance than the other tables. 

There appear to be a few differences between ratings by respondents from the three levels 
of engagement. Three of the five factors rated as most important to least engaged schools’ sustained 
participation in Quest concern logistical matters, such as time and funding. Participants from 
moderately engaged schools rated among the top five most important factors two variables 
concerning vision. Two of the five variables rated most important to highly engaged schools’ 
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continued involvement concern the match between the school and Quest, and two others concern 
Quest itself. 



Table 13 

Factors Rated Most Important to Highly Engaged Quest Schools’ Sustained Participation 



Factor 


N 


Mean 


SD 


Ways Quest supports school improvement (rallies, co- ventures, etc) 


25 


4.80 


.41 


Relevance of Quest to the school 


24 


4.79 


.41 


Personal beliefs about education and change 


25 


4.76 


.52 


Match between school and Quest network goals 


23 


4.70 


.56 


Quest processes (Data in a Day, Protocol process, etc.) 


25 


4.68 


.48 



Personal beliefs about education and change remained an important factor to respondents 
from all three levels of engagement when asked of their relevance to sustained association with 
Quest. It is also interesting to note that for ratings concerning both initial and sustained 
involvement, respondents from least engaged schools gave lower ratings of importance to factors 
than did those from moderately engaged schools, who in turn assigned lower ratings than did those 
from highly engaged schools. 

Tests of Statistical Significance 

T-tests and analyses of variance (AN OVA) were conducted in order to explore whether 
differences in mean ratings ofthe importance of factors to schools’ initial and sustained involvement 
in Quest were due to actual differences between the groups under investigation or were due to 
sampling error. In other words, t tests and ANOVAs were run to answer the questions, "Are 
elementary and high school Quest network participants likely from the same population or not?" and 
"Are respondents from the least, moderately, and highly engaged Quest schools likely from the same 
population or not?" Because of the exploratory nature of the study, two-tailed t tests and ANOVAs 
were used: Quest staff did not have hypotheses concerning the directionality of differences. 

It should be noted that the assumptions of these two statistical tests were violated in this 
study. The sample was not random, nor was it assumed that the data were drawn from a normally 
distributed population or that the samples had homogenous variances. Phillips (1982) contends, 
however, that "since those assumptions now appear to be far less important than originally thought, 
the recent trend toward increasing use of distribution-free tests is currently being reversed" (p. 1 39). 
Likewise, Glass and Hopkins (1984) report research suggesting that violation of the assumptions of 
normality and homogeneity of variance has little impact upon the robustness of t tests and ANOVAs. 
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To examine this claim. Quest staff conducted Mann- Whitney U and Kruskal- Wallis tests (although 
even these nonparametric tests assume samples are random) in addition to t tests and ANOVAs: 
Results were quite similar. Administration of the Mann- Whitney U test located three additional 
statistically significant differences as well as those found via t tests. The Kruskall-Wallis test 
replicated ANOVA results and located one additional statistically significant difference. The 
findings from t tests and ANOVAs are reported here; statistically significant nonparametric results 
are presented in Appendix E. Nonetheless, all findings should be interpreted with caution, given that 
the samples in the study are not random. 

Again due to the exploratory nature of this study, the significance level was set at . 10, rather 
than the conventional .05 or .01. With a significance level of .10, Quest staff might leam of 
potentially important differences between the groups under investigation that they might not access 
with lower levels of significance. For Quest staff, the purpose of statistical tests of significance in 
this study is to identify differences for future, more rigorous exploration. 

Statistically significant differences were found between elementary and high school 
respondents for only 6 of the 48 variables (see Table 14). Statistically significant differences were 
located between elementary and high school participants’ ratings of the importance of three factors 
to their schools’ initial participation in Quest. Elementary participants rated the significance of the 
amount of time needed to do improvement work at their schools to their schools’ first decision to 
become involved in Quest with a mean of 4. 16 (SD 1 .04) on the 5-point scale, whereas high school 
respondents gave a mean rating of 3.48 (.93). With a t value of 2.438, the between-group difference 
in means was statistically significant at the .05 level. Elementary school respondents rated the 
importance of the amount of time available to them for professional development to their schools’ 
initial participation in Quest with a mean of 4.00 (SD 1.13), while the mean rating for high school 
participants was 3.45 (SD 1.01). The t value of 1.812 was statistically significant at the .10 level. 
For the variable concerning the importance of the vision for learning in schools to schools’ initial 
association with Quest, elementary respondents had a mean of 4.57 (SD .68), and high school 
respondents had a mean of 3.79 (SD 1 .35). With a t value of 2.564, this difference was statistically 
significant at the .05 level. 

Statistically significant differences were also found between elementary and high school 
respondents’ ratings of the importance of three variables to their schools’ sustained participation in 
Quest. Statistically significant differences were again evident for the variables concerning the 
amount of time needed to do improvement work at the school and to individuals for professional 
development. Elementary school respondents gave a mean rating for the importance of the amount 
of time needed to do improvement work at their schools to their schools’ continued participation in 
Quest of 4.39 (SD .70), with high school participants giving a mean rating of 3.96 (SD .98). The 
t value for this difference was 1.950, which was significant at the .10 level. For the variable 
regarding the amount of time available to individuals for professional development, elementary 
respondents gave a mean rating of 4.22 (SD 1 .07), and high school respondents gave a mean rating 
of 3.68 (SD 1.13). With a t value of 1 .772, the between-group difference was statistically significant 
at .10. A third statistically significant difference was found between elementary and high school 
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respondents’ ratings of the importance of the ways Quest supports school improvement, such as 
through rallies and co-ventures, to their schools’ ongoing participation in the network. The mean 
rating for elementary respondents was 4.61 (SD .61), while the mean rating for their high school 
counterparts was 4.08 (SD .95). With a t value of 2.557, the difference was statistically significant 
at .05. 

Table 14 

Statistically Significant Quest Involvement TTest Results by School Level 



Factor influencing initial involvement 
in Quest 


Level 


N 


Mean 


SD 


SE 

Mean 


t 

Value 


df 


Prob. 


Amount of time needed to do 


Elem. 


31 


4.16 


1.04 


.19 


2.438 


50 


.018** 


improvement work at school 


High 


21 


3.48 


.93 


.20 








Amount of time available to individuals 


Elem. 


31 


4.00 


1.13 


.20 


1.812 


51 


.076* 


for professional development 


High 


22 


3.45 


1.01 


.22 








Vision for learning in the school 


Elem. 


30 


4.57 


.68 


.12 


2.564 


32.19$ 


.015** 




High 


24 


3.79 


1.35 


.28 









Factor influencing sustained 
involvement in Quest 


Level 


N 


Mean 


SD 


SE 

Mean 


t 

Value 


df 


Prob. 


Amount of time needed to do 


Elem. 


33 


4.39 


.70 


.12 


1.950 


54 


.056* 


improvement work at school 


High 


23 


3.96 


.98 


.20 








Amount of time available to individuals 


Elem. 


32 


4.22 


1.07 


.19 


1.772 


52 


.082* 


for professional development 


High 


22 


3.68 


1.13 


.24 








Ways Quest supports continuous school 


Elem. 


33 


4.61 


.61 


.11 


2.557 


56 


.013** 


improvement (rallies, co-ventures, etc.) 


High 


25 


4.08 


.95 


.19 









* Statistically significant at the .10 level. 

** Statistically significant at the .05 level. 

$ Degrees of freedom are not represented as whole numbers in this instance because, according to Levene’s test for 
homogeneity of variance, groups rating this item did not possess equal variances. SPSS automatically adjusts the 
degrees of freedom accordingly. 

ANOVAs were conducted to compare the variance between and within groups ofrespondents 
based upon the level of their schools’ involvement in Quest. A total of 12 statistically significant 
differences were found at the exploratory .10 level. Post hoc tests were conducted to determine 
among which of the three groups the statistically significant differences were located. The Scheffe 
test was used for groups determined by Levene’s test for homogeneity of variance to have equal 
variances, while Dunnett’s C was used for those determined not to have equal variances. 
Statistically significant ANOVA results are presented in Table 15. 

Five statistically significant differences were located with respect to factors impacting 
schools’ initial participation in Quest. With an /’ratio of 2.773, statistically significant at the .10 
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level (F probability of .072), respondents from highly engaged Quest schools rated the importance 
of the scheduling of Quest events to their schools’ initial involvement in the project more highly than 
did their moderately engaged colleagues. Participants from highly engaged schools rated the 
significance of personal beliefs to their schools’ first association with the network statistically 
significantly more highly than participants from the least engaged schools (F ratio of 4.169, F 
probability of .021). Respondents from highly involved schools rated the importance of the match 
between school and Quest network goals higher than did respondents from both moderately and least 
engaged schools (F ratio of 4.835, F probability of .013). The ways Quest supports school 
improvement was rated statistically significantly higher by those from highly involved schools than 
by those from least involved schools (F ratio of2.796,/ r probability of .071). Last, participants from 
highly engaged schools rated statistically significantly higher than their counterparts from least 
engaged schools the importance of Quest processes to their schools’ initial involvement in the 
project (F ratio of 3.323, improbability of .045). 

Seven statistically significant differences were located with regard to factors influencing 
schools’ sustained participation in Quest. With anFratio of 3. 643, statistically significant at the .05 
level (^probability of .033), respondents from highly involved schools rated higher the importance 
of the scheduling of Quest events to their schools’ ongoing participation than did their colleagues 
from both moderately and least engaged schools. With an F ratio of 5.083, statistically significant 
at the .01 level (improbability of .009), those from the least involved Quest schools rated more lowly 
than their moderately and highly engaged counterparts the importance of the vision for learning in 
their schools to their continued participation. Both the relevance of Quest to the school (F ratio of 
7.165, F probability of .002) and the match between school and Quest network goals (F ratio of 
5.496, F probability of .007) were rated statistically significantly higher by participants from the 
most engaged schools than by those from either moderately or least engaged schools. The match 
between school and Quest network beliefs was rated higher by those from highly involved schools 
than by those from the least involved schools (F ratio of 3.1 17, F probability of .053). With an F 
ratio of 8.040, statistically significant at the .001 level (F probability of .001), respondents from 
highly engaged schools rated higher than their colleagues from moderately engaged schools the 
importance of the ways Quest supports school improvement to their schools’ ongoing participation. 
Respondents from highly engaged schools rated statistically significantly higher than those from 
moderately engaged schools, who in tum rated higher than those from the least engaged schools, the 
importance of Quest processes to their schools’ continued involvement in the project (F ratio 10.306, 
F probability of .0001). 
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Table 15 

Statistically Significant Quest Involvement ANOVA Results by Level of Engagement 



Factor influencing initial involvement in Quest 


df 


F ratio 


F 

probability 


Significant 
differences by 
group 


Scheduling of Quest events 


2 


2.773 


.072* 


high > moderate 




49 








Personal beliefs about education and change 


2 


4.169 


.021** 


high > low 




51 








Match between school and Quest network goals 


2 


4.835 


.013** 


high > moderate, 




43 






low 


Ways Quest supports school improvement (rallies, 


2 


2'796 


.071* 


high > low 


symposia, etc.) 


50 








Quest processes (Data in a Day, Protocol process, 


2 


3.323 


.045** 


high > low 


etc.) 


45 

















Significant 


Factor influencing sustained involvement in Quest 


df 


F ratio 


F 


differences by 






probability 


group 


Scheduling of Quest events 


2 


3.643 


.033** 


high > moderate, 


53 






low 


Vision for learning in the school 


2 


5.083 


.009*** 


low < moderate. 


55 






high 


Relevance of Quest to the school 


2 


7.165 


.002*** 


high > moderate, 


50 






low 


Match between school and Quest network goals 


2 


5.496 


007*** 


high > moderate, 




49 






low 


Match between school and Quest network beliefs 


2 


3.117 


.053* 


high > low 




48 








Ways Quest supports school improvement (rallies. 


2 


8.040 


.001**** 


high > moderate 


symposia, etc.) 


55 








Quest processes (Data in a Day, Protocol process. 


2 


10.306 


0001***** 


high > moderate 


etc.) 


52 






>low 



* Statistically significant at the .10 level. 

** Statistically significant at the .05 level. 

*** Statistically significant at the .01 level. 

**** Statistically significant at the .001 level. 

***** Statistically significant at the .0001 level, 



CONCLUSIONS AND RECOMMENDATIONS 



Conclusions 

A variety of conclusions may be drawn based upon the data. Focus group and interview data 
are most suggestive of overall themes important to schools’ involvement in reform, whereas the 
survey data allow for somewhat more detailed comparisons of themes between elementary and high 
school participants, and between respondents from least, moderately, and highly involved schools. 
Conclusions from the qualitative and quantitative data diverge, in part, because of differences in the 
methodologies themselves. For instance, respondents offered different assessments of the factors 
important to their schools’ participation when asked to do so in an open-ended format than when 
they were requested to rate predetermined factors. 

Qualitative Conclusions 

Focus group and interview data clearly suggest that administrative support for involvement 
in an externally facilitated school improvement endeavor is a key component to schools’ initial and 
sustained participation. Such support appears to be important in two major ways. First, central 
office and building-level administrators have the capacity to secure funding, substitute coverage, and 
released time to enable the involvement of school community members in the reform effort. 
Respondents in this study saw a clear link between administrative support and access to such 
resources. Second, active administrative support is vital to the success of improvement work faculty 
choose to undertake at their schools. Respondents noted that passive or merely verbal support of 
their efforts by administrators was not sufficient to convince others in the school to accept or 
champion reform. 

Relatedly, the availability of funding to support school participation in reform appears to be 
an important furtherance to ongoing involvement. Respondents from more involved schools reported 
their awareness of how important money was to their participation, while those from less involved 
schools noted that its lack impeded their continued involvement. 

Also clearly significant to school staffs’ initial and continued involvement in school 
improvement reform is the relevance of the reform to schools’ concerns, goals, and beliefs. The 
overlap of school issues and content to be offered by Quest piqued school staffs’ original interest in 
participation. Similarly, continued relevance helped sustain their involvement in the network over 
time. Participants from schools that had left the network or were uncertain about their continued 
involvement, conversely, reported that they questioned the relevance of Quest to their schools’ 
interests. 

Schoolwide consensus for involvement in a school reform effort appears to be an important 
contribution to initial participation. Likewise, school willingness to accept feedback and institute 
change seems to influence schools’ ongoing engagement with reform. Respondents in this study 



thought that without such openness to self-analysis and transformation, schools would not participate 
in Quest in the first place, let alone over time. 

Ongoing participation in reform seems to be hindered by the competing priorities school 
community members must manage at their schools. These range from daily school functions to 
administering before- and after-school programs to involvement in additional school reform proj ects. 
The variety of efforts school staff undertake results in feelings of overcommittment, lack of time to 
participate in other endeavors, and a sense of staff shortage to accommodate all the work. 

Time is also a concern to participants in ongoing school improvement work. First, finding 
the time to attend project events away from school is an issue, particularly as such excursions 
involve acquiring substitute coverage, completing make-up grading and other work, and sacrificing 
time from additional school commitments and from families. And second, unearthing time for 
reflecting, planning, and implementing specific reform initiatives is a challenge for school 
community members whose days are already filled with regular school duties. 

Logistical concerns were especially important to those from small schools, who thought that 
finding time and resources to engage in sustained school reform was a greater challenge due to their 
fewer staff. On the other hand, respondents from larger schools noted that they faced similar issues. 

Interestingly, it appears that, for some respondents, positive prior experience with AEL 
services influenced their schools’ initial decision to participate in the Quest project. In other words, 
the organizational reputation facilitated involvement. 



Quantitative Conclusions 

Survey data provide a different perspective on the issues impacting initial and sustained 
participation in school improvement endeavors, allowing comparisons between elementary and high 
school respondents and between those from least, moderately, and highly engaged network schools. 

Neither focus group nor interview data indicated that personal beliefs about education and 
change were particularly important to schools’ involvement in improvement efforts. However, mean 
ratings from survey data clearly suggest that all participants surveyed thought such beliefs to have 
been most important to their schools’ first and continued engagement with Quest. 

School factors, such as the vision for learning and the level of communication within a 
school, importantly influence schools’ initial decision to participate in a school reform project such 
as Quest. But the relevance of Quest to the school and the nexus of school and network goals are 
also important to initial involvement. 

Sustained involvement is supported slightly differently by various factors. The relevance of 
Quest to the school becomes more important. A characteristic of the project itself, how it supports 



school improvement, becomes meaningful, as does the availability of funding to attend network 
meetings. The match between school and project goals remains important. 

Quest staff had hypothesized that high schools would confront different challenges to initial 
and ongoing participation in the project than elementary schools. Differences in mean ratings 
indicate several differences in this study. In addition, high school respondents tended to rate the 
importance of factors to their schools’ initial and sustained involvement somewhat lower than their 
elementary school counterparts, although the gap lessened slightly with regard to ongoing 
participation. 

Differences in mean ratings suggest a divergence in the importance of various factors to 
elementary versus high schools’ initial participation in the project. School factors, such as the vision 
for learning, amount of time available to do improvement work, and level of communication with 
the school, appear to have been most important to elementary schools’ first decision to become 
involved in Quest. High school respondents, on the other hand, report that issues concerning the 
relevance of the project to their schools was most important to initial involvement. 

Personal beliefs about education and change were nearly equally highly important to 
elementary and high school participants’ schools’ sustained participation. Relevance of the project 
to the school and school beliefs appeared to become somewhat more important to elementary schools 
when respondents were asked to assess their ongoing involvement. And characteristics of Quest 
itself, such as the amount of information received about the project and project processes for needs 
assessment and reflection, seemed to become slightly more important to high schools’ continued 
participation. 

Quest staff also predicted that schools with various levels of involvement with the project 
might find different factors important to their participation. And mean differences in ratings of 
factors do suggest several divergent concerns. While personal beliefs about education and change 
were important to schools from all three levels of engagement and their initial participation, they 
were most important to respondents from moderately and highly engaged schools. The vision for 
learning in the school, on the other hand, was most important to the original participation of those 
from the least engaged schools. Nonetheless, the vision for learning in a school and personal beliefs 
about education and change were among the most important contributing factors to schools’ initial 
participation from all three levels of involvement. 

Time to attend project events and do improvement work were also important factors in the 
least engaged schools’ initial involvement in Quest. Respondents from moderately engaged schools 
were more concerned with school factors, including the vision for learning and the level of 
communication within their schools, as well as the vision of building-level leaders and the amount 
of time available to individuals for professional development. Differently, those from highly 
engaged schools were most attentive to issues of relevance when initially considering participation 
in Quest; three of the five factors most highly rated by these respondents concerned the nexus 
between project and school foci. 



As with initial participation, personal beliefs about education and change continued to be 
important contributors to continued participants across the three levels of engagement. However, 
whereas such beliefs had been most important to highly and moderately involved schools in terms 
of their initial participation, they became most important to those from least and moderately engaged 
schools’ ongoing involvement. 

There were several other differences as well. School-level factors were most important to 
least and moderately engaged schools’ sustained association with Quest. For least engaged schools, 
these included logistical matters such as the amount of time to do improvement work at school and 
the availability of funding. School vision for learning and building-level vision were important for 
moderately engaged schools’ ongoing involvement. Respondents from highly engaged schools were 
most concerned with the relevance of Quest to their schools and characteristics of Quest itself. 

Respondents from elementary and high schools were statistically significantly different from 
each other in six of their ratings of the importance of various factors to their schools’ participation 
in Quest. In terms of initial participation in the project, the amount of time needed to do 
improvement work at school, the amount of time needed to attend Quest events, and the vision for 
learning in the school were statistically significantly more important to elementary school 
respondents than to their high school counterparts. The amount of time needed to do improvement 
work and to attend network events were also statistically significantly more important to participants 
from elementary schools, as were the ways Quest supports school improvement, such as through 
rallies and co-ventures. It thus appears that elementary school respondents are likely truly different 
from their high school colleagues in terms of the importance of these six factors to their schools’ 
initial participation in the project. 

Twelve statistically significant differences were located among respondents from schools at 
three levels of engagement with Quest. Five of these pertained to factors influencing schools’ initial 
involvement in the project. The scheduling of project events was statistically significantly more 
important to highly involved schools than to moderately involved schools. Highly engaged schools 
appear to have been more concerned with the match between school and Quest network goals than 
either their moderately or least engaged counterparts. And personal beliefs about education, the 
ways Quest supports school improvement, and Quest processes were more important to highly 
engaged schools than to the least engaged schools’ original participation. 

The seven remaining statistically significant differences concerned schools’ sustained 
participation in the network. The scheduling of Quest events, relevance of Quest to the school, and 
the match between school and network goals were more important to highly involved schools than 
to either moderately or least engaged schools. In terms of Quest processes, highly engaged schools 
found them more important to sustained involvement than did moderately engaged schools, who in 
turn reported them more important than did the least engaged schools. The least engaged schools 
found less important to continued participation than either their moderately and highly involved 
counterparts the vision for learning in their schools. The match between school and Quest network 
beliefs was more important to respondents from highly engaged than least engaged schools, and the 



ways Quest supports school improvement was more important to highly than moderately engaged 
schools. 



Recommendations 

Recommendations are offered to Quest staff based upon the conclusions drawn above. 

First, Quest staff should acknowledge that some factors are beyond their purview in terms 
of recruiting and maintaining the participation of some network schools. These include school 
issues, such as the level of communication within a school and school vision for learning. Similarly, 
school community members’ personal beliefs about education and change are particularly beyond 
the reach of project staff influence during recruitment. 

Nonetheless, because relevance of the project to schools contributes so importantly to initial 
and sustained school participation, Quest staff should be alert to the variety of ways their project 
might intersect with each schools’ goals, concerns, and beliefs in spite of other school factors outside 
their purview. To take advantage of such intersections fully, project staff may consider gathering 
some relevant information about potential network schools prior to recruitment, should the project 
continue through additional iterations. 

In order to maintain continued school interest and engagement, Quest staff should also 
consider conducting ongoing assessment of the goals and concerns network schools have throughout 
their participation. Although the project cannot attempt successfully to meet all the needs of all 
network schools, it can continually align its work with school interests or make serious attempts to 
do so. 



Relatedly, because of the flexibility and non-directive nature of the project, Quest is 
especially well-suited to ensure that any network endeavor does not add to the burdens of schools 
already attempting to manage a variety of competing priorities. Continuous monitoring of individual 
schools’ undertakings may provide project staff the information they need to integrate Quest and 
school efforts. 

Given the importance of active administrative support to schools’ involvement in the project, 
Quest staff should consider strategies to secure, nurture, and maintain it over time. Some 
participants reported that their prior knowledge of AEL influenced their decision to participate in 
the Quest project; exploiting the "power of weak ties" is one way of courting potential network 
schools. Another strategy might include using knowledge of the ways Quest and school goals 
overlap to recruit schools, and then ensuring continued overlap to encourage ongoing participation. 
Offering individualized support to administrators throughout the project may also invite their 
sustained engagement. Alternatively, Quest staff might consider, in possible further iterations of the 
project and under optimum circumstances, admitting to the network only those schools that appear 
to have strong, unambiguous support from building-level administrators, who in turn have similar 
support from their central offices. 



Schoolwide consensus for involvement in Quest also appears quite important to schools’ 
participation. Project staff might consider instituting methods similar to those used in other school 
reform efforts to ensure that the majority of those in network school communities agree to become 
involved. These methods include requiring potential project school communities to vote via secret 
ballot for or against participation. Quest staff have requested network schools to sign a cooperative 
agreement, but they did so after approximately 1 8 months of project activity. Such a strategy may 
be more appropriate at the beginning of project work. 

Quest staff should be alert to differences in the factors elementary and high school network 
respondents thought to be important to their schools’ involvement in the project. For instance, time 
to attend project events and to conduct improvement work at school seem to be more important to 
elementary school participation than to high school. During recruitment, staff might consider 
offering schools strategies for finding such time. Continued consultation on the matter of time may 
also prove helpful to elementary schools throughout the project. In terms of sustained participation, 
the ways Quest supports school improvement, such as via rallies and co-ventures, are important to 
elementary schools; project staff should continue offering appealing venues for such schools. 

Elementary and high school differences in mean ratings of the importance of variables to 
their involvement suggest that high schools may be more concerned with external factors than 
elementary schools, such as the overlap between Quest and school goals and Quest processes. 
Elementary schools, on the other hand, appear somewhat more concerned, especially in terms of 
initial involvement, with issues internal to the school. (The relevance of the project to elementary 
schools does become more important with regard to continued participation, however.) 
Consequently, project staff should examine the possibility of recruiting and supporting elementary 
and high schools differently. 

In terms of offering support to schools that have been least involved in the project, difference 
in mean ratings suggest that they may be somewhat more concerned with logistical matters than 
other schools. Given that administrative support is seen as central to brokering time and funding, 
Quest staff should give some attention to strategies for ensuring that administrators recognize the 
need for their action in terms of resolving logistical issues. Alternatively, project staff might offer 
less engaged schools more support or technical assistance in terms of methods for finding time and 
funding. 

Relevance and the match between school and network goals and beliefs appear to contribute 
significantly to highly involved schools’ participation. Quest staff should therefore maintain their 
focus on assisting schools reach their own goals. In addition, project staff should be alert to the ways 
characteristics of Quest itself are received by network participants. Quest processes, for instance, 
are most important to those from highly involved schools, less so for moderately involved schools, 
and even less so for the least engaged schools. 
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Much more, and more rigorous, investigation of what influences participation in school 
reform needs to be conducted. Further study, for instance, ought to include randomization 
techniques to bolster generalizability and ensure the appropriateness of inferential statistical 
methods. A larger study including participants in other reform efforts might yield important insights 
about the significance of various factors to schools operating in other reform contexts. 
Substantively, Quest staff may want to examine the relationship between school level and degree 
of involvement, and whether an interaction between the two influences the importance of various 
factors to participation; the small sample size in this study precludes a valid investigation of this. 
Also, project staff should reconsider the scale used and explore the reasons elementary schools and 
highly engaged schools tended to have higher ratings overall than other respondents. It is possible v 
that respondents interpreted the scale in terms of whether or not their schools possessed each of the 
factors rather than in terms of the level of importance of each to their involvement. Another avenue 
for exploration might include factor analyses to examine, for instance, whether the variables 
concerning relevance and the match between project and school goals and beliefs in fact represent 
one underlying construct. 
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Quest for Quality Learning Communities 

A Program for Continuous School improvement 



School improvement is challenging work; to be 
effective, it must be continuous. Improvement is 
not a single act or program; it is a process of al- 
ways wanting to learn more about how better to 
help all students achieve at higher levels. Improve- 
ment is visionary; it involves risk-taking, uncer- 
tainty, and a rejection of “doing what we’ve al- 
ways done.” Most of all, improvement re- 
quires more than individual effort: 
it is a collaborative endeavor that 
engages and responds to the 
diverse voices within an 
entire community. 

Teams from 20 
schools in a four- 
state region now 
collaborate with 
staff from the Ap- 
palachia Educa- 
tional Laboratory \ 

(AEL) to study and ^ 
learn together in the 
Quest project, and 
each school takes a 
slightly different path. For 
example, one school targets 
increased parent involvement; 
another hopes to raise the level of stu- 
dent thinking through teachers’ working 
together and coaching one another; a high school 
improves teaching by listening to what students 
say about how they learn best; other schools fo- 
cus on specific curriculum areas such as writing 
or science education. 

The Quest framework unifies their thinking about 
school improvement. These core values offer a 
blueprint for continuous progress: ongoing ques- 
tioning of practice, high expectations for all, in- 



dividual responsibility for better performance, col- 
legial sharing and support, and thoughtful reflec- 
tion on practice. 

Stemming from these values is a clearly defined 
vision of student excellence that is shared by all 
members of the school community. A strong 
learning culture encourages both students and 
teachers to choose continuous im- 
provement as a way of life in their 
school. Members of the 
school community con- 
nect to one another 
through a shared com- 
mitment to im- 
proved learning 
conditions for all. 
» Shared leadership 
encourages and en- 
s ables everyone to 
assume responsibil- 
ity for making a 
positive impact on 
the school commu- 
nity. Shared goals for 
student learning moti- 
vate individuals to improve 
their performance and help fo- 
cus the energies of the entire com- 
munity. The collection, analysis, and use 
of student assessment data sustains continuous 
improvement, providing a measure of the effec- 
tiveness of the community’s efforts. SMART 
learners are Successful, Motivated, Autonomous, 
Responsible, and Thoughtful. Fully equipped to 
become lifelong learners, they are ready for life 
and work in the 21st century. In short, continu- 
ous improvement spawns the energy and excite- 
ment necessary to transform a collection of indi- 
viduals into a true learning community. 
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Goals of the Quest Project 



1 . Connect with colleagues. By serving on a Quest leadership team, participants connect with others on their school 
team, forming bonds that enhance working relationships. In addition, Quest teams connect with teams from 
other schools, districts, and states, allowing everyone to learn from others’ experiences. A listserv, inquiry@ael.org, 
facilitates connections across the network. 

2. Create a learning community. Teams become part of the Quest net- 
work learning community with the expectation of recreating this ex- 
perience in their own community. 

3. Connect with concepts and stories related to continuous school im- 
provement. At Quest rallies, the Quest framework is a source of study, 
dialogue, and sharing among teams. 

4. Create personal and shared meaning. The Quest network places a 
high value on processes such as reflection and dialogue, which lead to 
deeper understandings of continuous improvement. 

3. Commit to continue learning with this community. Quest schools 
have made a three-year commitment to study and learn together, with 
a focus on improving student achievement. 

6. Commit to continue the Quest back home. The “rubber hits the road” at schools, not at Quest events. AEL helps 
school teams take their learnings home and apply them for the benefit of students. Site visits, called Co-Ventures 
in Learning, provide opportunities for AEL staff to visit each school, in order to better understand the context of 
that schools efforts, and tailor assistance to the school’s needs. 



What Is a learning community? 
“Learning communities are essen- 
tially communities of inquirers . . . 
sustained by a continued commit- 
ment to share this journey of 
exploration with one another on 
matters people care deeply about” 
(Ryan, 1995). 



Peter Senge et ai. (1 994) write that a 
learning organization “is a place 
where people continually expand 
their capacity to create the results 
they truly desire, . . . and where 
people are continually learning how 
to learn together.” 
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The Quest project hopes to achieve results at three different levels: 

• For individuals, sharing leadership on a Quest team leads to more reflective practice and renewed under 
standing of the concepts that support continuous improvement. 

• For schools, Quest will provide motivation and support for ongoing and/or new school-based initiatives to 
improve teaching and learning. 

• For the Quest network of schools, our collaborative learning and research will yield stories, insights, 
processes, and products — all of which will be helpful to the broader educational community. 
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For more information about 




contact Sandra Orletsky or 
Beth Sattes (sattesb@ael.org) at 

m n 

P. O. Box 1348, Charleston, WV, 25323 
800-624-9120 

\www.aeI.org/ rel/quest/index/htm^ 
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Engagement Study 
Focus Group Protocol 

1 . How would you describe your school ’ s level of involvement with Quest? What does involvement 
in Quest require of your school? 



2. When your school was first considering becoming involved with the Quest project, what factors 
or issues influenced your school’s decision? 



A. Specifically, what school factors seemed to be most important in determining the 
decision to be involved in Quest? 



B. And what characteristics or factors of Quest seemed to be most important in 
determining your school’s decision to be involved in Quest? 



3. It is sometimes challenging to stay involved over time in a large project. 



A. What about your school helps your school stay involved with Quest? Gets in the way 
of your staying involved? 



B. What about Quest itself helps your school stay involved? Gets in the way of staying 
involved? 



(Optional probes) 

4. What most attracted your school to being involved with Quest? What most attracts your school 
now? 



5. What was least attractive to your school about being involved with Quest when you first began? 
What is least attractive now? 
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Phone interview protocol 

Hello. This is (name) with the Appalachia Educational Laboratory’s Quest project. May I please 
speak with (name)? 

Hello. This is (name) with the Appalachia Educational Laboratory’s Quest project. We mailed 
you a letter about two weeks ago requesting that you talk with us briefly. Did you receive it? 

If no: We are studying what helps or gets in the ways of schools’ participation in a 
network like Quest. I have several questions I’d like to ask about this. This should only 
take about 5 or 10 minutes. 

If yes: As the letter explained, we are studying what helps or gets in the ways of schools’ 
participation in a network like Quest. I have several questions I’d like to ask. This should 
only take 5 or 10 minutes. 



And is now a good time for you to speak with me? 

If no: When would be a better time for us to talk? <record reply> I will call you then. 
Thank you! 

If yes: Before we begin, I’d like to assure you that anything we discuss will remain 
anonymous and confidential. Because all the data I collect will be aggregated, nothing 
you say will be attributable to you in any way. Do you have any questions? 

Let’s begin . . . (ask interview questions and record answers) 

I also have a brief survey I’d like to ask you to take. I could administer it to you over the phone 
or I could fax it to you, and ask you to return fax it. Which would you prefer? 

If fax: Could I have your fax number, please? 

If over phone: I am going to read a list of factors to you. Please rate how important each 
one was to, first, your school’s initial decision to get involved in Quest, and second, to 
your school’s involvement over time. The scale is 1 to 5. 1 is "very unimportant," 3 is 
"neither unimportant nor important," and 5 is "very important." Do you have any 
questions? 

(Read each factor, circling responses as you go. Prompt if necessary to specify initial and 
sustained involvement.) 

We have finished. Thank you for talking with me. We appreciate your input very much. 
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Phone interview protocol 
School: 

Interviewee role: 

1. Please describe what school factors seemed to be most important in determining your 
school’s decision to be involved in Quest. 



2. And what characteristics or factors of Quest seemed to be most important in determining 
your school’s decision to be involved in Quest? 



3. It is sometimes challenging to stay involved over time in a large project. What about your 
school has gotten in the way of your school’s staying involved? (If school is still involved, ask 
what about the school helps them stay involved.) 



4. What about Quest itself has gotten in the way of your school’s staying involved? (If school 
still involved, ask what about Quest helps them stay involved.) 
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APPENDIX D: 



Quest Engagement Survey 
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APPENDIX E: 



Tables of Statistically Significant Quest Involvement Results 
from Nonparametric Tests 



Table 16 

Statistically Significant Quest Involvement Mann- Whitney U Test Results 



Factor influencing initial 










Mann- 


Wilcoxon 


z score 




involvement in Quest 


Level 


N 


Mean 


SD 


Whitney 

U 


W 




Prob. 


Amount of time to do 


Elem. 


31 


4.16 


1.04 


175.000 


406.000 


-2.988 


.003*** 


improvement work at 
school 


High 


21 


3.48 


.93 










Amount of time available to 


Elem. 


31 


4.00 


1.13 


230.500 


483.500 


-2.083 


.037** 


individuals for professional 
development 


High 


22 


3.45 


1.01 










History of prior reform 


Elem. 


30 


3.93 


1.48 


168.500 


321.500 


-2.008 


.045** 


efforts at the school 


High 


17 


3.24 


1.30 










Vision for learning in the 


Elem. 


30 


4.57 


.68 


236.500 


536.500 


-2.371 


* 

* 

oo 

o 


school 


High 


24 


3.79 


1.35 










Quest processes (Data in a 


Elem. 


28 


3.75 


1.27 


203.000 


413.000 


-1.666 


.096* 


Day, Protocol process, etc.) 


High 


20 


3.25 


1.07 











Factor influencing 
sustained involvement in 
Quest 


Level 


N 


Mean 


SD 


Mann- 

Whitney 

U 


Wilcoxon 

W 


z score 


Prob. 


Amount of time to do 


Elem. 


33 


4.39 


.70 


277.000 


553.000 


-1.892 


.059* 


improvement work at 


High 


23 


3.96 


.98 










school 


















Amount of time available to 


Elem. 


32 


4.22 


1.07 


243.500 


496.500 


-2.024 


.043** 


individuals for professional 


High 


22 


3.68 


1.13 










development 


















Vision for learning in the 


Elem. 


33 


4.45 


.97 


295.000 


620.000 


-2.047 


.041** 


school 


High 


25 


4.04 


1.10 










Ways Quest supports school 


Elem. 


33 


4.25 


.61 


282.500 


607.500 


-2.280 


.023** 


improvement (rallies, co- 


High 


25 


4.08 


.95 










ventures, etc.) 



















* Statistically significant at the . 10 level. 

** Statistically significant at the .05 level. 

*** Statistically significant at the .01 level. 
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Table 17 

Statistically Significant Quest Involvement Kruskall- Wallis Test Results 2 



Factor influencing initial involvement in Quest 


df 


Chi- 

square 


Prob. 


Scheduling of Quest events 


2 


5.580 


.061* 


Personal beliefs about education and change 


2 


5.734 


.057* 


Match between school and Quest network goals 


2 


9.243 


. 010 *** 


Match between school and Quest network beliefs 


2 


5.684 


* 

CO 

o 


Ways Quest supports school improvement (rallies, co-ventures, etc.) 


2 


5.860 


.053* 


Quest processes (Data in a Day, Protocol process, etc.) 


2 


6.831 


.033** 



Factor influencing sustained involvement in Quest 


df 


Chi- 

square 


Prob. 


Scheduling of Quest events 


2 


5.685 


.058* 


Vision for learning in the school 


2 


6.264 


044 ** 


Relevance of Quest to the school 


2 


9.456 


.009*** 


Match between school and Quest network goals 


2 


11.397 


.003*** 


Match between school and Quest network beliefs 


2 


6.314 


.043** 


Ways Quest supports school improvement (rallies, symposia, etc) 


2 


13.721 


0QJ**** 


Quest processes (Data in a Day, Protocol process, etc) 


2 


12.276 


. 002 *** 



* Statistically significant at the .10 level. 

** Statistically significant at the .05 level. 

*** Statistically significant at the .01 level. 
**** Statistically significant at the .001 level, 



2 SPSS does not permit the use of post hoc tests following the Kruskall- Wallis test. 

Group differences are therefore not reported here. However, because the results from this test are 
almost identical to those from the ANOVAs presented earlier, one might assume differences 
located by the Scheffe and Dunnet’s C tests conducted thereafter to be identical as well. 
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APPENDIX F: 



Completed Evaluation Standards Checklist 



J 
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Citation Form 



Program Evaluation Standards (1994, Sage) guided the development of this (check one): 

request for evaluation plan/design/proposal 

evaluation plan/design/proposal 

evaluation contract 

evaluation report 

X other: evaluation/research repor t 

t o interpret the information provided on this form, the reader needs to refer to the full text of the standards as they appear in Joint 
ommittee on Standards for Educational Evaluation, The Program Evaluation Standards (1994), Thousand Oaks, CA, Sage. 



The Standards were consulted and used as indicated in the table below (check as appropriate): 



1 

^Descriptor 

U1 Stakeholder Identification 


The Standard was 
deemed applicable 
and to the extent 
feasible was taken 
into account. 

X 


The Standard was 
deemed applicable 
but could not be 
taken into account. 


The Standard was 
not deemed appli- 
cable. 


Exception was taken 
to the Standard. 


Bj2 Evaluator Credibility 


X 








HU3 Information Scope and Selection 


X 








U4 Values Identification 


X 








b(J5 Report Clarity 


X 








Bl)6 Report Timeliness and Dissemination 


X 








u7 Evaluation Impact 


X 








FI Practical Procedures 


X 








■F2 Political Viability 






X 




Bf3 Cost Effectiveness 


X 








PI Service Orientation 


X 








■P2 Formal Agreements 


X 








|p3 Rights of Human Subjects 


X 








P4 Human Interactions 


X 








_ P5 Complete and Fair Assessment 


X 








■P6 Disclosure of Findings 


X 








™P7 Conflict of Interest 


X 








P8 Fiscal Responsibility 


X 








Bai Program Documentation 


X 








BA2 Context Analysis 


X 








A3 Described Purposes and Procedures 


X 








aA4 Defensible Information Sources 


X 








■A5 Valid Information 


X 








A6 Reliable Information 


X 








A7 Systematic Information 


X 








Ba 8 Analysis of Ouantitative Information 


X 








B A9 Analysis of Oualitative Information 


X 








A10 Justified Conclusions 


X 








flAII Impartial Reporting 


X 








■ At 2 Metaevaluation 


X 








B Na me Caitlin Howley-Rowe Date: 7/16/99 

■ (typed).- o • i . . (y 

1 Position or Title: Researoh Assist' ^ BEST COPY AVAILABLE 

m Agency: AEL , Inc . 

■ Aririress- P.0. Box 1348 Charleston, WV 25325 
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lc D\r 1 10 Document: Author of document 

(e.g., author of document, evaluation team leader, external auditor, internal auditor) 
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